Sr Data Engineer | Full Time
PB Consulting
New York, NY
Posted On: Jun 04, 2024
Posted On: Jun 04, 2024
Job Overview
Salary
Depends on Experience
Required Skills
- Scala
- Python
- Apache Spark
- PySpark
- AWS
- SQL
Job Description
Roles and Responsibilities
- Develop products and services on the latest technologies through contributions in development, enhancements, testing, and implementation.
- Work on migrating applications from an on-premises location to the cloud service providers.
- Develop, modify, and extend code for building cloud infrastructure, and automate using CI/CD pipeline.
- Partners with business and peers in the pursuit of solutions that achieve business goals through an agile software development methodology.
- Work with peers across the system to define and implement best practices and standards.
- Assess applications and help determine the appropriate application infrastructure patterns.
Required Experience/Skills
- Hands-on experience in building/implementing cloud platforms/applications on the AWS platform.
- Experience in developing data pipeline solutions to ingest and exploit new and existing data sources.
- Expertise in leveraging SQL, programming languages like Python, and ETL tools like Databricks.
- Expertise in AWS Compute (EC2, EMR), AWS Storage (S3, EBS), AWS Databases (RDS, DynamoDB), AWS Data Integration (Glue).
- Good understanding of AWS Identify and Access management, AWS Networking, and AWS Monitoring tools.
- Proficiency in CI/CD and deployment automation using the GITLAB pipeline.
- Proficiency in one or more programming languages e.g., Python, Scala.
Job ID: PC240203